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Abstract 

Using the Pathfinder Scaling Algorithm (Pathfinder) (Interlink, 1994), the researchers conducted a cognitive task analysis 
expertise in Human Performance Technology (HPT). The study investigated: I) the extent to which Pathfinder-derived coherence 
scores were associated with other measures of HPT expertise; 2) how HPT experts organize their knowledge of the discipline; 
and 3) how experts organize their HPT knowledge differently than novices. Findings include: 1) a significant correlation between 
coherence and the number of HPT-related books participants had written; 2) a Pathfinder -derived concept map of HPT; and 3) 
expected novice/expert differences in Pathfinder similarity and relatedness scores. 

Most representations of academic disciplines have been created when experts depict or report what they know. Experts draw 
diagrams, establish competencies, and write textbooks. Clearly, this approach works; new practitioners use these representations 
to enter and gain competency in the discipline. Some practitioners eventually master elements of the discipline. However, there 
are potential problems that can arise when practitioners rely on expert self-report. Simply stated, experts are often unable to state 
what they know and do that makes them experts. Two cognitive processes account for this phenomenon. First, experts may be 
unable to explicitly state knowledge they have learned implicitly. Implicit learning occurs when people acquire knowledge about 
the relationships comprising a complex system, without necessarily knowing in advance what the variables are. Examples include 
solving differential equations underlying a simulated parking task (Broadbent, Fitzgerald, & Broadbent, 1986) and predicting 
where a light will appear on a computer screen in a serial learning task (Willingham, Nissen, & Bullemer, 1989). In these 
situations, people have encountered a complex task, observed the variables in operation in an unselective manner, and attempted 
to store the contingencies among them (Berry, 1993). Essentially, people implicitly learn these complex system and the 
relationships comprising it by “mucking about.” One of the results of implicit learning is that the resulting knowledge is 
relatively inaccessible to free recall (Berry, 1993). Experts who have implicitly constructed domain-specific knowledge may not 
be able to articulate what they know. 

Second, experts may be unable to articulate cognitive processes that have reached a level of automaticity. According to 
Anderson (1993), this situation exists when people explicitly learn new skills or knowledge by using language to mediate a 
cognitive production, such “determine the cause of a performance gap” or “select an appropriate intervention to address the cause 
of the gap.” With successive tuning, the production fires automatically, without invoking any mediating language. The result is 
automatic, fluent performance — without the ability to report on the cognitive aspects of the performance itself 

One way to avoid potential problems associated with expert self-report is to employ cognitive task analysis methods. The 
Pathfinder Scaling Algorithm is one such method (Schvaneveldt, 1990; Interlink, 1994). This software program transforms 
participants’ pair-wise ratings of related concepts into a semantic network — a concept map comprised of nodes (the concepts) 
and links that depict the relationships among the concepts. 

A Pathfinder-derived concept map of a discipline could supplement existing representations created using expert self-report. 
One opportunity for using Pathfinder to map a discipline lies in the area of human performance technology (HPT): “. . .a set of 
methods and procedures, and a strategy for solving problems, for realizing opportunities related to the performance of people. It 
can be applied to individuals, small groups, and large organizations” (ISPI, 2001). As no cognitive task analysis of HPT has been 
conducted to date, this study sought answers to the following three research questions: 

1 . To what extent are Pathfinder-derived coheraice scores associated with other measures of HPT expertise? 

2. How do experts in the field organize their knowledge of HPT? 

3. To what extent do experts organize their HPT knowledge differently than novices? 
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Literature Review 

This section defines key Pathfinder concepts and addresses the literature relevant to each of the study’s research questions. 
In addition to generating concept maps that represent domain knowledge without the complications associated with expert self- 
report, Pathfinder generates three measures employed in this study: coherence, relatedness, and similarity. Table 1 describes these 
measures. 



Table 1 

Definition, Uses, and Examples of Coherence, Relatedness, and Similarity Scores (adapted from Villachica, 2000) 



Measure 


Definition 


Uses 


Examples 


Coherence 
(Theoretical 
Range = -1.0 to 
+1.0) 


A Pearson Product -Moment 
correlation indicating the 
internal consistency of 
ratings within an individual’s 
or group’s set of concept 
ratings (Interlink, 1996). 


A coherence score for an 
individual’s or group’s set of 
concept ratings calculated. 
This score is compared to 
that of other individuals. 


Student coherence scores 
obtained prior to instruction are 
compared to those obtained after 
instruction to determine if 
learning occurred. 


Relatedness 
(Theoretical 
Range = -1.0 to 
+1.0) 


The Pearson Product - 
Moment correlation between 
sets of concept ratings 
(Interlink, 1996). 


The comparison of an 
individual’s or group’s set of 
concept ratings to another’s. 


The proximity matrices of 
students are correlated with a 
referent matrix containing the 
averaged responses of a panel of 
experts. 


Similarity 
(Theoretical 
Range = 0 to 
+1.0) 


The proportion of shared 
links in two concept maps 
using the same terms. The 
ratio of shared 
neighborhoods in two 
Pathfinder networks 
(Interlink, 1996). 


The Pathfinder networks of 
an individual or group are 
compared to those of another. 


The Pathfinder networks of 
students are compared to their 
instructor’s. The resulting 
similarity scores are used to 
predict performance on a test. 



Coherence and Expertise 

Since the knowledge structures of experts are more organized than those of novices, experts in a given domain should possess 
relatively higher coherence scores. The higher the coherence score, the greater the internal consistency of the concept ratings. 
Four studies have investigated the relationship between coherence and expertise in different domains. However, results have been 
uneven, indicating a need for additional replication. 

Three of these studies found a correlation between coherence and expertise. In a classroom setting, Housner, Gomez, and 
Griffey (1993) investigated the extent to which coherence scores and other knowledge structures would predict performance in a 
physical education preservice course. They found a moderate correlation between coherence and a simulated teaching activity (r 
= .63). Gaultieri, Fowlkes, and Ricci (1996) investigated the effectiveness of training that eight Navy and Air Force pilots 
received over a five-day workshop. This training consisted of pre-briefings, simulated missions, and debriefings. The pilots were 
divided into two teams of four pilots each, with each participants rating concept pairs on the first, third, and fifth days. Results of 
a repeated measures analysis of variance revealed a statistically significant increase in participants’ coherence scores over time 
(f2,14 = 1 1.55. p. < .001). In a similar study, Stout, Salas, and Kraiger (1997) studied 12 naval aviator trainees enrolled in a one- 
day, complex training program addressing aviation teamwork and communication skills. The researchers reported that course 
attendees earned higher mean coherence scores than a control group who received no training (t = 2.70, £ < .01, M.traineees== -63, 

I^ontrol ~ *26). 

One study did not find a relationship between coherence scores and domain expertise. Dorsey, Campbell, Forste, and Miles 
(1999) used Pathfinder networks to create concept maps that evaluated relationships generated by 88 computer users and 
compared them against the scores of four subject matter ecperts. Their results indicated that coherence scores were not 
significantly related to any of the concept map scores. 

Within the domain of HPT, coherence scores would conceivably be positively correlated with several measures of expertise. 
Experts in HPT are members of a community of practice that has its roots in academe. As a result, experts can be expected to 
present at conferences, write articles, and author books. They can be expected to have spent at least 10 years developing their 
expertise (Ericsson, 1996). 

Organization of HPT Domain Knowledge 

The organization of HPT domain knowledge can be viewed in terms of existing expert representations of the domain and the 
extent to which a Pathfinder-generated concept map could supplement these existing models. 
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Existing expert representations of HPT. Given the depth and breadth of HPT, a variety of experts have authored conceptual 
and procedural representations of the discipline. Each representation depicts an expert’s organization of his or her HPT expertise. 
Conceptual representations place HPT within theoretical contexts and illustrate the relationships among its components. 
Procedural representations illustrate the steps comprising an HPT process. 

Stolovitch and Keeps (1999) provide both conceptual and procedural representations of HPT. In their conceptual model, the 
authors depict external and organizational environments that influence internal requirements related to human performance. Once 
articulated, these requirements trigger behaviors that produce accomplishments, which are subjected to verification. 
Accomplishments that are aligned with business requirements are accepted; those that are not aligned are subjected to subsequent 
alteration in behaviors, which change the organization’s accomplishments. Their procedural representation of HPT consists of an 
iterative, 10-step process that begins with the identification of business requirements and concludes with monitoring and 
maintaining performance interventions. At this point, the process can repeat itself, leading to the identification of new 
requirements and subsequent interventions. 

Addison (2001) provides a conceptual view of HPT in his Performance Consultant HPT Landscape. This representation of 
the discipline depicts the interaction of two conceptual and two procedural dimensions that comprise the landscape: 

1 . Levels of the environment, ranging from the worker to society (conceptual); 

2. Principles of performance technology (conceptual); 

3. Systematic approach, starting with need and ending with evaluate (procedural); and 

4. System(s) view point, beginning with conditions and ending with feedback (procedural). 

The International Society for Performance Improvement has created its own procedural representation of HPT. This process 
model focuses on a systematic combination of performance analysis, cause analysis, and intervention selection (ISPI, 2001). The 
majority of steps comprising the HPT model contain additional conceptual information providing additional detail about a given 
step. 

Pathfinder representations of expertise. In addition to the conceptual and procedural representations used to depict expert’s 
organization of HPT, one can use Pathfinder networks to represent this domain-specific expertise. The end result of Pathfinder 
analysis is a concept map that depicts a semantic network representing a domain, such as HPT. These concept maps are intended 
to represent the knowledge structures that humans store in their minds (Jonassen, Beissner, & Yacci, 1993). 

Researchers have employed Pathfinder-derived concept maps to study the nature of expertise in some 1 3 studies and nine 
different domains, ranging from aircraft combat to programming to electronic troubleshooting to medicine (Villachica, 2000). In 
a similar vein, Pathfinder-derived concept maps could depict the way in which HPT experts organize their domain knowledge, 
providing another representation of this complex discipline. 

Novice/Expert Differences in the Organization of HPT Knowledge 

It is not surprising that novices and experts exhibit different levels of domain-specific performance. A well-established line of 
research traces the sources of these performance differences to the organization of cognitive structures in memory. De Groot 
(1978) and Chase and Simon (1973) demonstrated that expertise in chess is partially attributable to the organization of memory. 
Chess masters possess more highly organized and complex structures in longterm memory than chess experts, who possess more 
organized and complex structures than non-experts. These differences in domain knowledge allow chess masters to employ a 
larger visual scan than chess novices (Reingold, Chamess, Pomplun, & Stampe, 2001). Similarly, Chi, Feltovich, and Glaser 
(1981) report that physicists sort physics problems differently than novices, with experts’ organizational strategies revealing more 
sophisticated cognitive structures, based upon their knowledge of the “deep structure” of the domain. 

Pathfinder-related studies that address novice/expert differences in domain-specific cognition have found that experts tended 
to exhibit greater degrees of intragroup agreement than novices. That is, the relatedness, similarity, and coherence scores of 
experts tended to be more similar than those of novices. For example, Schvaneveldt, Durso, Goldsmith, Breen, and Cooke (1985) 
conducted a discriminant analysis using Pathfinder measures that successfully predicted novice and expert performance in a 
fighter pilot task. Schvaneveldt, Durso, and Dearholt (1989) employed Pathfinder-produced concept maps to study differences in 
the ways in which biology graduate students (“experts”) and undergraduate students (“novices”) organized their knowledge of 
biology. Schvaneveldt, Beringer, Lamonica, Tucker, and Nance (2000) used Pathfinder to demonstrate differences in the 
priorities that novice and experienced commercial aircraft pilots assign to information viewed during the phases of a flight. 
Thompson (1992) employed Pathfinder-based measures to reveal differences in the organization of domain knowledge among 
expert, non-expert, and novice nurses. 

Given the consistency of these findings, one should expect to find novice-expert differences in the organization of HPT 
knowledge. That is, the Pathfinder-related measures of an operationally defined set of experts should be more like each other than 
they are like those of a set of novices. 

Methodology 

Participants and Procedure 

ISPI issued invitations to participate in the study to approximately 4,500 of its 6,000 members (75 percent). ISPI published 
the invitations on two occasions, the first invitation in its online newsletter ISPI Quick Read . After 14 days, only 38 people had 
responded to this invitation. ISPI then emailed a second invitation to its members two weeks later. With this invitation, another 
103 persons had responded, making a total of 141. Of those, 4 people had submitted duplicate data sets, which were subsequently 
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removed from the study. Of the 137 people who had completed the survey, 73 went on to complete all 435-concept ratings. Thus, 
the overall response rates to the study were 2 and 1 percent, respectively. 

The researchers drafted an initial list of 50 HPT-related concepts based upon a review of the Handbook of HPT (Stolovitch & 
Keeps, 1999). Acting as the operationally defined referent experts for the study, the ISPI 2000-2001 Board participated in a 
modified Delphi process that resulted in a final list of 30 HPT concepts that would be employed in the study. ISPI members were 
invited to participate in the study via two methods. One, a general announcement was published in the ISPI Quick Read 
newsletter, and second, individual invitations were emailed. The second section of both the ISPI Quick Read article and the email 
invitation contained an informed consent form. Subjects who agreed to participd:e in the study indicated their agreement by 
clicking on a link that directed them to a website used to collect demographic data and concept ratings. In order to maintain 
confidentiality, all data were aggregated into a single computer file using the computer to assign unique, anonymous participant 
identification codes. Qnce subjects had entered their responses into an online dataset, all names were deleted from the file, 
thereby guaranteeing the anonymity of all participants and protecting subjects against unintentional disclosure outside the 
experiment. 

The website consisted of the several pages, which participants completed in order. After viewing a welcome page that 
described the purpose of the study, participants viewed instructions about rating HPT concepts. The practice component 
comprised of the next two pages of the web site. In the first page, participants then viewed a list of practice terms, which 
appeared next to radio checkboxes. Participants could click next to any term they did not know. In the next page, participants 
completed a set of practice ratings. The survey component of the web site consisted of a single web page that participants 
completed to provide information about themselves. The ratings component of the web site consisted of two pages. In the first 
page, participants viewed the complete list 30 HPT concepts; with the option to check any terms they did not know. In the next 
page, participants rated up to 345 randomly presented pairs of HPT concepts, checking on the radio buttons comprising a rating 
scale to assign lower numbers (1-4) to unrelated pairs and higher numbers (6-9) to related pairs. Participants assigned a “5” to 
concept pairs they did not know or could not rate. The last pages of the web site thanked participants and allowed them to view 
additional information about the HPT Research Group conducting the study. The concept ratings participants provided were then 
stored as proximity matrices for subsequent Pathfinder analysis. 

Results 

Coherence and HPT Expertise 

Table 2 summarizes the measures, survey items, number of respondents, means, standard deviations, and standard errors used 
in the regression analysis of coherence scores. 

Table 2 

Descriptive Statistics Used in the Regression of Coherence Scores on Measures of HPT Expertise 



Measure 


Item 


Statistics 


Coherence scores 


Not applicable 


N = 


73 






M = 


0.324 








0.153 


Years HPT practitioner 


How many years have you been a 


N = 


137 




practitioner of human performance 


M = 


11.161 




technology? 




8.239 


Number of juried 


How many JURIED presentations relating 


N = 


137 


presentations 


to HPT have you written or co-written? 


M = 


3.321 






SD = 


11.758 


Number of non-juried 


How many NQN-JURIED presentations 


N = 


137 


presentations 


relating to HPT have you written or co- 


M = 


4.409 




written? 


SD = 


8.839 


Number of juried 


How many JURIED articles relating to HPT 


N = 


137 


articles 


have you written or co-written? 


M = 


0.744 






SD = 


3.097 


Number of non-juried 


How many NQN-JURIED articles relating 


N = 


137 


articles 


to HPT have you written or co-written? 


M = 


2.007 






SD = 


6.026 


Number of book 


How many book chapters relating to HPT 


N = 


137 


chapters 


have you written or co-written? 


M = 


0.774 






SD = 


3.132 


Number of books 


How many books relating to HPT have you 


N = 


137 




written or co-written? 


M == 


0.197 






SD = 


1.028 



To determine the relationship between Pathfinder coherence scores and other measures of expertise, the researchers 
employed a multiple linear regression analyses that regressed coherence scores on the number of years as an HPT practitioner. 
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number of juried presentations, number of non-Juried presentations, number of juried articles, number of non-juried articles, 
number of book chapters, and number of books the participant had either written or co-written. To control for Type I error, alpha 
was set at .05. All variables were entered at once. The multiple linear regression produced statistically significant results (r = 
.458, £ = .026), accounting for 21 percent of the variance in coherence scores. As depicted in Table 3, the only statistically 
significant independent variable contained in the equation was the number of books the participant had written or co-written. The 
other independent variables did not reach levels of statistical significance. The magnitude of this correlation is weak, although it 
approaches the moderate threshold of .5. 

Table 3 

Regression Analysis Summary for HPT Expertise Variables Predicting Coherence Scores 



Variable 


Coherence (n = 73, if = . 0,210)* 


B 


SEB 


3 


Years HPT practitioner 


0.0006 


0.002 


0.027 


Number of juried presentations 


-0.0005 


0.007 


-0.012 


Number of non-juried presentations 


-0.0049 


0.007 


-0.149 


Number of juried articles 


0.0056 


0.029 


0.033 


Number of non-juried articles 


0.0098 


0.013 


0.126 


Number of book chapters 


-0.0025 


0.016 


-0.020 


Number of books 


0.2747 


0.067 


0.468* 



p<0.05 



It should be noted, however, that only 9 of 137 participants who responded to this item on the demographic survey had 
written a book. Stevens (1992) notes that statistical procedures based upon regression are mathematical maximization procedures 
may capitalize on chance and artificially increase alpha, especially when the number of subjects is not large relative to the 
number of variables. Thus, the results obtained in this study may not replicate in another sample. 

Organization of HPT Expertise 

As a measure of the internal consistency of a participant’s concept ratings, coherence scores can be used as an operational 
indicator of domain expertise (Interlink, 1994). To identify the participants who would act as the expert HPT practitioners in this 
study, the researchers identified all participants with coherence scores greater than 0.4 (h = 21, M = .4976, SD = .1309) 
(Schvaneveldt, personal communication, June 23, 1998). The researchers then averaged the concept ratings of these operationally 
defined experts and subjected the results to Pathfinder analysis, which produced the following concept map. Figure 1 represents 
the organization of a statistically derived, “averaged expert’s” cognitive organization of HPT. 
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The key concept in the map is represented by "'results,^' which is linked to four other sets of concepts. The first set branch of 
concepts contains the terms ""learning theory, ""ISDfi ""needs analysis ""job and task analysis ""systematic^ "" measurement 
"" evaluation and ""feedback'' The organization of these concepts indicates that experts employ ISD and its related components to 
obtain results. The second branch of concepts related to “ results" contains the terms ""goals, ""change management," ""motivation," 
""performance barriers," ""cause analysis," ""performance support," and ""interventions," These terms form the basis of HPT 
theory and reflect its behavioral roots. The organization of these concepts indicates that experts employ HPT theory to obtain 
results. A third branch of concepts related to ""results" contains the terms ""Return on Investment" , ""business case," and 
""requirements analysis." Experts use requirements analysis to build business cases from which they can predict a return on 
investment that quantifies obtained results. The organization of these items indicates that experts employ business-related 
measures to obtain results. A fourth branch of concepts related to ""results" is more complex and consists of two separate subsets. 
The first subset consists of the terms ""outputs," ""inputs," ""conditions, ""work organization," and ""Systemic." Systemically 
accounting for workplace organization, experts assess the conditions of performance, their inputs, and their oatputs in 
determining performance results. It is also important to note that ""workplace organization" is also linked to ""business case," 
indicating that experts use the context of the workplace to create such cases. The second subset consists of the terms 
""information," ""resources," human capital, ""competencies," ""individual and team workers," and ""collaboration." The 
organization of these terms indicates that collaboration among individual and team workers builds human capital, which can be 
described in tmtis of competencies. Human capital is linked to resources. Information about resources allows resources to be 
used as inputs to performance-which ultimately lead to outputs and results. 



Novice/Expert Differences in the Organization of HPT Knowledge 

To explore differences in the organization of HTP knowledge among an operationally defined set of novices and experts, the 
researchers employed a median split. In this instance, experts were operationally defined as any participant possessing a 
coherence score one standard deviation above the mean (n = 8, M ~ .1035, SD = .0553). Novices were operationally defined as 
any participant possessing a coherence score one standard deviation below the mean (n — 8, M = .1353, SD — . 0161). Table 4 
summarizes means and standard deviations associated with experts’ and novices’ similarity and relatedness scores. 



Table 4 



Numbers, Means, and Standard Deviations for Novice and Expert Similarity and Relatedness Scores 
Group Measure of Knowledge Organization 

Similarity Scores Relatedness Scores 



N 

Expert 8 

Novice 8 



M SD N 

0.1353 0.0553 7 

0.1035 0.0161 8 



M SD 

0.0659 0.0421 

0.0322 0.1122 



To determine if the similarity and relatedness scores of experts were more similar to each other than they were to novices, the 
study employed a multivariate analysis of covariance (MANCOVA). Novice/expert designation acted as the independent 
variable. Similarity and relatedness scores acted as the dependent variables. As coherence scores were used to operationally 
define groups and represented a source of variation beyond the control of the researchers, they were used as a covariate in the 
analysis. To control for Type I error, alpha was set to .05. 

The results of the MANCOVA indicated that the group effect was statistically significant (Wilks’ Lambda = .544, f(l , 14) = 
4.615, 2 .= .035). Eta-squared (1 -Wilks’ Lambda) revealed that approximately 46 percent of the variance in the linear combination 
of the dependent variables was associated with group differences. As summarized in Table 5, tests of between-subjects effects 
revealed that group differences were attributable to similarity scores alone. 

Table 5 

MANCOVA Results for Similarity and Relatedness Scores as a Function of Group and Coherence 



Source Dependent Variable df SS MS f 



Covariate 


Relatedness 


1 


0.0035 


0.0035 


.439 


(Coherence) 


Similarity 


1 


0.0065 


0.0065 


5.820* 


Group 


Relatedness 


1 


0.0065 


0.0065 


.816 




Similarity 


1 


0.0110 


0.0110 


9.803* 


Residual 


Relatedness 


12 


0.0952 


0.0079 






Similarity 


12 


0.0135 


0.0011 




Total 


Relatedness 


15 


.137 








Similarity 


15 


.249 







*p<0.05 



Discussion 

Coherence and HPT Expertise 

Although the number of years participants had been a HPT practitioner was expected to weakly correlate with participants’ 
coherence scores, it did not. Although the sample exceeded Ericsson’s (1996) threshold of ten years to reach expert status in a 
domain, no correlation with coherence scores or any other measure of expertise was found. Since the number of non-juried/juried 
presentations and publications are based upon recognition in the field and the results of peer review, it was thought that these 
indirect measures of HPT expertise would correlate with coherence scores. Although these measures correlated among 
themselves, they did not correlate with coherence; book production alone correlates with coherence. 

Three possible explanations for the significant relationship between coherence scores and authoring an HPT-related book 
include: 1) desire to create detailed conceptual structures or schemas, 2) tolerance for tedium, and 3) low sample size. The 
correlation between coherence scores and writing an HPT-related book could be the result of the capacity or desire of an author 
to develop highly detailed and organized cognitive structures or schemas. Books are representations of schemas, divided into 
chapters and sections. Perhaps book authors possess such schema, making it more likely that their cognitive maps would correlate 
with coherence scores. 

Another conceivable reason for the correlation between book authorship and coherence scores may be book authors’ 
tolerance for tedious activity. Authoring depends on patience, self-discipline, and a large devotion of time and energy. Authoring 
also requires one to switch attention between small details and large-picture views frequently. The structure of this study’s pair- 
wise rating activity required the same sort of mental activity for an extended period of time (between 30 minutes and one hour). 
Perhaps individuals who were able to complete the ratings and stay on task for this time are also individuals who can author 
books. 

Finally, the low sample size employed in the study may have precluded obtaining significant results. In addition to artificially 
increasing alpha, low sample size may have lacked adequate statistical power to detect otherwise significant relationships 
between coherence scores and measures of HPT expertise. Replicating this study with a larger sample and modifications to the 
website to decrease experimental mortality (that is, the number of participants who dropped out of the study between completing 
the information page and their concepts) could produce less ambiguous findings. 

Organization of HPT Expertise 

The expert concept map is similar to other expert representations of HPT, suggesting the convergent validity to this finding. 
The information and resource components of the human capital branch of the concept map roughly correspond to the 
organizational environment of Stolovitch and Keeps’ (1999) conceptual representation of HPT. The business case branch 
corresponds to their components of business goals/objectives and internal requirements. Where Stolovitch and Keeps focus on 
accomplishments and their verification, the expert concept map focuses on results. Additional similarities can be found 
comparing the expert concept map to Stolovitch and Keep’s procedural representation of HPT. The business case branch of the 
concept map addresses Stolovitch and Keeps’ steps for identifying business and performance requirements. Likewise, the HPT 
branch of the concept map addresses the steps “define performance gaps, specify gap factors, and select interventions.” 

One of the underlying principles of the HPT landscape model (Addison, 2001) is a focus on results and outcomes. The expert 
concept map also communicates a focus on results, which is the key concept on the map. Sections related to ISD, human capital, 
business cases, and HPT are linked to obtaining results. The systems and business case portions of the Addison model are similar 
to the ISD systems and business case branches of the expert concept map. The predictive validity of the expert concept map could 
be determined by determining the extent to which it predicts performance in HPT. For example, concept ratings could be 
obtained from previous winners of ISPI awards. These scores could be compared against a control group who did not win such 
awards. Subsequent statistical analysis could determine the extent to which Pathfinder-derived relatedness, similarity, and 
coherence scores predicted the award- winners. In addition to providing a unique perspective of HPT as a discipline, practitioners 
and researchers could employ the concept map in different ways. For practitioners, the expert concept map can assist in the 
creation of HPT competencies. The different branches of the map could represent the major HPT competencies. The organization 
of subordinate and related competencies could also be derived from the concept map. 

Experienced practitioners could use the concept map as a “mind tool” to introduce new practitioners to HPT. Specifying the 
nature of the links could help learners construct their knowledge of the discipline (Jonassen, 2000). The similarity measure that 
Pathfinder derives from the concept map could also be used as an assessment tool for practitioners and researchers alike. As new 
practitioners gain experience solving increasingly difficult problems in the domain, their similarity scores should increase, 
indicating that their mental map of HPT is increasingly like those of experienced practitioners. Their development within this 
community of practice could be modeled statistically, with different stages of practitioner HPT development lending themselves 
to different learning and other types of performance supporting interventions. If such stages and interventions could be 
determined, then whole programs could be designed to help novices move along a continuum of HPT expertise, potentially 
improving the quality of HPT practice while decreasing the time required obtaining expert-like performance. 

Novice/Expert Differences in the Organization of HPT Knowledge 

The similarity scores of experts are higher than those of novices, indicating that expert concept maps are more like those of 
other experts than of novices. This finding replicates the results of other studies comparing the organization of expert and novice 
cognition. Specifically, experts share a greater proportion of the links in their individual concept maps with those of the 
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“averaged expert’s” concept map (p. 8) than do novices (14 percent versus 10 percent, respectively). It should be noted, however, 
that the small sample used in this analysis may not provide replicable results. 

Conclusion 

While the results obtained in this study are intriguing and suggest potential uses of the expert concept map and related 
Pathfinder-based measures in HPT, these results are based upon a small sample. Replication with a larger sample is certainly 
warranted. 
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